This is the revision history for web scraping in <matrix.org:#rAnyijhRxzFRfdjSHl:matrix.org>

factoid_idsubjectcopulapredicateauthornamespace
15884 web scraping is try WWW::Mechanize or Mojo::UserAgent (or Test::Mojo in tests) or if JavaScript is involved Selenium or Playwright thrig <*:##NULL>
11531 web scraping is HTTP::Tiny (with HTTP::CookieJar), LWP (with HTTP::Cookies), WWW::Mechanize, Mojo::UserAgent (with Mojo::CookieJar), cURL, wget (and bindings). JavaScript interaction might require Selenium, automated Firefox or PhantomJS. su-shee <*:##NULL>